The 360-degree video, also known as immersive video is an emerging technology, which can provide “feeling as sensation of present”. The sense of immersion is achieved by surrounding a user with wrap-around scene covering a panoramic view, in particular, 360-degree field of view. The “feeling as sensation of present” can be further improved by stereographic rendering. Accordingly, the panoramic video is being widely used in Virtual Reality (VR) applications.
Immersive video involves the capturing a scene using multiple cameras to cover a panoramic view, such as 360-degree field of view. The immersive camera usually uses a set of cameras, arranged to capture 360-degree field of view. The set of cameras may consist of as few as one camera. Nevertheless, typically two or more cameras are used for the immersive camera. All videos must be taken simultaneously and separate fragments (also called separate perspectives) of the scene are recorded. Furthermore, the set of cameras are often arranged to capture views horizontally, while other arrangements of the cameras are possible.
The set of cameras have to be calibrated to avoid possible misalignment. Calibration is a process of correcting lens distortion and describing the transformation between world coordinate and camera coordinate. The calibration process is necessary to allow correct stitching of videos. Individual video recordings have to be stitched in order to create one 360-degree video. Stitching of pictures has been well studied in the field via the context of blending or seam processing.
FIG. 1 illustrates an example of images from panoramic videos corresponding to a given time instance. The panoramic videos are captured using four cameras, where the principle axis of each camera is rotated roughly 90° from that of a neighboring camera. The set of four non-stitched images 110 consists of four images (112, 114, 116 and 118) from four cameras. Each camera covers very wide field of view (i.e., using wide angle lens) so that pictures from neighboring cameras have a substantial overlapped area. The set of pictures corresponding to the panoramic videos at a given instance are then stitched to form a pre-stitched picture 120. A pre-stitched picture 120 is a stitched picture that is stitched prior to enter the video compression system for subsequent compression.
For panoramic video, in particular, the 360-degree video, multiple videos may be captured using multiple cameras. A large amount of bandwidth or storage will be needed for the data necessary to render a full virtual reality environment. With the ever increasing video resolutions, the required bandwidth or storage becomes formidable. Therefore, it is desirable to develop efficient video compression techniques for the 360-degree video, in particular the pre-stitched panoramic video.